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ABSTRACT 

The first of two section' of this paper describes a 
pilot program that used interactive video (IAV) technology to enhance 
the scripting skills of teacher certification assessors in the 
administration of a classroom observation technique. A comparative 
analysis was drawn between three groups of assessors: those who 
received IAV programed instructional training, video only training, 
and no training at all. A pretest was administered to the first two 
groups; all three groups received posttests. No differences were 
found in the results for the three training methods, and this is 
attributed to the strong scripting skills already possessed by the 
assessors. The second section provides descriptions of each of the 
components of the IAV system used in the scripting practice study. 
The appendices contain examples of teacher certification criteria to 
be observed in the classroom, questionnaires used in the study, an 
example of a feedback screen, and a visual diagram of IAV components. 
(6 references) (DB) 
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USING INTERACTIVE VIDEODISC TECHNOLOGY 
TO ENHANCE ASSESSOR TRAINING 



The purposes of this paper are twofold. In the first section of the paper a pilot program 
will be described that used interactive videodisc technology to enhance the scripting 
skills of assessors in the administration of a classroom observation instrument. The 
second section of this paper will focus on interactive videodisc technology with a 
discussion about each component of the system. 

SECTION I: PILOT PROGRAM 

In Connecticut, for purposes of Hcensure, teachers take a series of examinations at 
various stages of teacher preparation and during their first year in the classroom. In this 
project we focused on the Beginning Educator and Support (BEST) assessment 
program that takes place during the teacher's first year. This assessment utilizes the 
Connecticut Competency Instrument (CCI) and involves six independent classroom 
observations conducted by six trained assessors. The CCI results in one of three 
outcomes; a teacher is granted a provisional teaching certificate; a teacher is 
recommended for a second year of support and assessment; or a teacher is not eligible 
to pursue licensure. 

The assessment begins with a preassessment interview, followed by a classroom 
observation period of 45 to 60 minutes when an assessor takes detailed notes (referred 
to as scripting). The scripting involves a comprehensive notetaking method designed 
to collect all relevant lesson content, instructional activities and classroom 
observations. A post-observation interview is then conducted which allows the 
beginning teacher an opportunity to clarify anything that occurred in the lesson that had 
not been previously explained. Following the observation, assessors categorize 
positive and negative evidence according to 22 defining attributes associated with 10 
scoring indicators. Each indicator is then rated as acceptable or unacceptable based on 
a set of anchors established during training. The indicators and their defining attributes 
are presented in Appendix A. 

Ind ; viduals participating as assessors attend over 70 hours of training and must pass a 
set of proficiency standards in order to be qualified to conduct assessments. 
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The development of the instrument, training of assessors and implementation of the 
program has been a large-scale and complex effort. For example, over the course of three 
years 1200 assessors have been trained in the state. Of this number, 450 proficient assessors 
evaluated over 700 beginning teachers during the 1990-91 school year. 

Considering the large cadre of assessors and the high stakes nature of this assessment, it is 
particularly important that assessors maintain the quality of their scripting and apply their 
ratings according to state standards. Ensuring that assessors maintain uniform standards is 
achieved by ongoing communications with assessors and annual large-scale follow-up 
training and calibration sessions. However, due to the labor intensity of individualized 
script evaluation and feedback, along with costs associated with the use of trainers, there 
have been few opportunities available to allow assessors to independently practice and 
sharpen scripting skills. 

Consequently, the primary purpose of this pilot study was to examine the feasibility of 
providing a training program that possessed the following characteristics: 
-Practice opportunities to maintain scripting proficiency 
-Practice that is individualized to meet the needs of each assessor 
-Practice that requires minimal human intervention without reducing effectiveness 
-Practice opportunities that result in high assessor acceptance 
-Practice that is administratively feasible, cost effective, and can be flexibly 
scheduled 

After evaluating several training options, we chose to develop a prototype training system 
based on a computer administered interactive videodisc system (IVD). Such a comouter 
administered system has the capability to systematically deliver video segments of 
classroom instruction (referred to as vignettes), and then interactively administer and score 
questions that test for scripting accuracy. An IVD approach appeared to not only address 
the desired characteristics noted above but could be designed to allow immediate and 
focused feedback. However, if an IVD system was to be considered in a statewide 
program, it was also important to determine if such a system could withstand the rigors of 
frequent use by a variety of assessors. 

In addition to the IVD system, we were interested in examining a "lower tech" and less 
costly approach to scripting practice. Accordingly, a small-scale study was conducted 
which compared an IVD approach to a conventional videotape presentation format. A 
description of this study follows. 



Method 



This study compared three groups of assessors; those who received IVD training, VIDEO training 
and CONTROL (no training). A pre-test was administered as part of the IVD and VIDEO training 
and all three groups received a post-test. A more detailed description of the method and procedures 
follow. 

Subjects 

All assessors that had recently attended update or new training in 1990 were invited to participate in 
a pilot study. In the invitation to assessors, the purpose of the study was described as an 
examination of the feasibility of using interactive videodisc workstations to review, practice and 
maintain the skills required in scripting. Of 700 invitations sent, 168 assessors expressed an interest 
in participating. 

From their initial training, evidence writing scores were available for 153 of the 168 potential 
subjects. Evidence writing scale scores range from 0 to 106 and measure the assessor's ability to 
effectively use a written script. Incomplete or inaccurate scripts result in lower evidence writing 
scores. The actual evidence writing scores for the 153 potential subjects ranged from 59 to 104 with 
a mean of 94.52. 

As an estimate of scripting ability, evidence writing scores were used as a basis for selecting the 62 
subjects used in the study. Accordingly, the 62 subjects were placed into one of the three 
experimental groups (IVD, VIDEO, CONTROL). To ensure that each group represented a balanced 
range of initial scripting abilities, subjects were distributed according to low (less than 91), medium 
(92-99) and high scores (greater than 100) on the evidence writing scale. 

As expected, this stratification procedure resulted in no significant differences between the three 
groups based on evidence writing scores, F=1.83, #=2,53, ^=.17. The mean scores were: IVD 
group, x=94.16; VIDEO group, x=95.68 and CONTROL group, x=93. 39. The mean across groups 
was 94.43. 

Fifty-five of the 62 subjects reported, as of the date of the post-test, that they had conducted between 
zero and 20 actual assessments. The mean number of assessments was 6.13 with a standard 
deviation of 5.15. There was no significant difference in the mean number of assessments conducted 
between the three groups, ^=0.55, d/=2,52, p=.58. The mean number of assessments were: IVD 
group, x=6.95; VIDEO group, x=6.22; and CONTROL group, x=5. 17. 
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Materials and Procedure 



Pretest 

This study consisted of a pretest for the IVD and VIDEO and a post-test for all groups (IVD, 
VIDEO, and CONTROL). For the IVD and VIDEO groups, the pre-test consisted of scripting two 
vignettes and responding to 33 multiple-choice questions based on the vignettes. The IVD subjects 
worked at one of four videodisc workstations. Each workstation consisted of a computer, monitor, 
keyboard, mouse, amplifier, speakers, headphones and a videodisc player. An administrator started 
the program and remained in the room during the session to observe and answer any questions that 
might arise. 

Since the IVD system utilized a mouse as the primary source of subject input and selection, the 
program began with a mouse practice exercise. As scripting practice began, participants were 
presented with introductory screens describing the task and asked to enter their name and social 
security number. The scripting practice exercise consisted of two components. 

In the first component, subjects were asked to script two vignettes. The first vignette consisted of 17 
minutes and the second vignette 1 1 minutes of classroom instruction. Following each vignette, 
subjects were allowed time to clarify and review their scripts. 

In the second component of scripting practice, a 33 item multiple-choice test was administered, 23 
items covered the first vignette and 10 items covered the second vignette. The questions were 
developed by a committee of assessor trainers and were designed to evaluate whether subjects 
accurately scripted the kinds of classroom evidence most important to the CCI. Subjects were given 
immediate feedback about their response to each question. If a subject incorrectly answered a 
question, the relevant portion of the vignette was replayed and subjects were prompted to follow 
along in their scripts. If the subject answered the same question wrong again, a second screen was 
displayed. In the upper left quadrant of this screen, participants were presented with the relevant 
video. On the bottom half of the screen, a transcript for that part of the video was provided. Finally, 

in the upper right quadrant war, a menu that allowed the user to control the video (stop, rewind, 
play), which page of the script they were observing, or to return to the question. Figure 1 illustrates 
the second feedback screen and Figure 2 illustrates the IVD scripting and questioning process. 

During the course of the IVD session, start and stop times, responses to each question and subject 
identification information were recorded for all participants. 



In the VIDEO condition, subjects assembled in two groups of approximately 10, and were asked to 
script the same two video vignettes that were viewed in the IVD condition. The video was presented 
on a single 20" monitor. After scripting the vignettes, subjects received the same 33 multiple-choice 
questions as used for the IVD group, but in paper-and-p^ncil format. After everyone completed the 
questions arid responses were collected, subjects were given an opportunity to review the answer 
key. 

The multiple-choice questions used in both the IVD and Video groups represented three types of 
questions. The first type was designed to measure assessor skill in recording verbatim evidence, the 
second was designed to measure the participant's practice of recording observational comments and 
the third was designed to examine the assessor's practice of scripting at critical points in the lesson. 
In both the IVD and VIDEO conditions, subjects were provided descriptions of the three types of 
questions after the multiple-choice questions were completed. Appendix B provides examples of 
each question type. 

Post-test 

Between two and three weeks after training took place the subjects in the IVD, VIDEO and 
CONTROL conditions were convened in groups of 20 to script a post-test vignette presented in 
videotape format on a 20" monitor. As with the VIDEO pretest, subjects were tested in small groups 
(n=20). After scripting the vignette, subjects were presented with 30 questions designed to assess 
their scripting accuracy. 

Following the pretest and again following the post-test, subjects were asked to evaluate each 
component of the IVD and VIDEO practice procedures. Appendix C provides the IVD, VIDEO and 
post-test questionnaires. 

Results 

Post-test performance of the three groups was not found to be significantly different, F=.80, d/=2,50, 
p=A5. Likewise no significant differences were found between the groups on the three question 
types: Verbatim Evidence, F=1.04, £//=2,50,p=.36; Observational Comments, F=.49, d/=2,50, 
p=.61; Critical Points in the Lesson, F=2.24, d/=2,50,p=.12. The means for these ANOVAS are 
shown in Table 1. The overall score was based on 30 items, of which, 15 items covered verbatim 
evidence, 9 items assessed observational comments and 6 items were related to critical points in the 
lesson. 
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Table 1 
Post-test Means 



Overall Verbatim Obser. Critical 



Group i Score Evidence Comments Points 

Mean SD Mean SD Mean SD Mean SD N 

CONTROL 23.72 2.87 11.83 1.58 6.72 1.27 5.17 0.92 18 

IVD 22.53 2.85 10.94 1.78 7.00 1.17 4.59 1.12 17 

VIDEO 23.06 2.67 11.39 2.09 7.11 1.18 4.56 0.86 18 



A second set of analyses was conducted comparing the IVD and VIDEO groups and using pretest 
scores as covariates. Again, no significant effects were found. Tables 2 and 3 summarize the results. 
The percentage of correct answers was used for these analyses. 



Table 2 
Analyses of covariance 







Groups 




Covariate 






F 


df 


P 


F 


df 


P 


Overall Score 


0.01 


1,28 


.908 


12.57 


1,28 


.001 


Verbatim Evidence 


0.34 


1,28 


.563 


5.37 


1,28 


.028 


Obser. Comments 


0.00 


1,28 


.988 


7.37 


1,28 


.011 


Critical Points 


0.30 


1,28 


.585 


1.72 


1,28 


.200 



Table 3 

Means on the Post-test Dependent Measures 



Overall Verbatim Obser. Critical 

Score Evidence Comments Points 



Mean SD Mean SD Mean SD Mean SD N 

IVD 0.75 0.10 0.72 0.12 0.78 0.13 0.76 0.19 17 

VIDEO 0.76 0.10 0.74 0.14 0.81 0.13 0.75 0.16 14 

To assess whether there were increases in scripting skills between the pretest and post-test, a 
repeated measures ANOVA was conducted comparing the IVD and VIDEO groups. Again, the 
percentage of correct answers was used instead of the number of items correct. For the purpose of 
these analyses, it was assumed that the items were of equal difficulty. 

As indicated in Table 4, no significant effects of Group or the Group by Time (pretest vs post-test) 

interaction were found. There was, however, a significant effect of Time on the overall score and 

observational comments measures. On overall scores, subjects significantly increased their 

percentage of correct answers between the pretest (x=0.71) and the post-test (x=0.76). 
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Likewise, on the observational comments questions, subjects significantly increases their percentage 
of correct answers between the pretest (x=0.61) and the post-test (x=0.79). 



Table 4 
Repeated Measure ANOVAS 



Group Time Group x Time 





F 


df 


P 


F 


df 


P 


F 




P 


Overall Score 


0.16 


1,29 


.69 


7.20 


1,29 


.01 


0.00 


1,29 


.92 


Verbatim Evidence 


2.07 


1,29 


.16 


1.40 


1,29 


.25 


3.47 


1,29 


.07 


Obscr. Comments 


1.66 


1,29 


.21 


45.72 


1,29 


.00 


0.79 


1,29 


.38 


Critical Points 


0.73 


1,29 


.73 


3.21 


1,29 


.08 


1.97 


1,29 


.17 



After both the VIDEO and IVD pretests, subjects were given a questionnaire designed to assess: 
1) the types of scripting opportunities subjects would most likely attend and, 2) the effectiveness of 
the various components of the pretest training. The scripting opportunities were rated on a five point 
scale ranging from l=unlikely to attend to 5=highly likely to attend. There was no significant 
difference between the VIDEO and IVD groups in their preference for a one-on-one refresher 
course with a BEST trainer, a small group refresher session or for videotape practice at home. There 
was, however, a significant difference in subjects preference for the type of training they had just 
experienced (r=3.36, d/=39, p=.0O2). Subjects in the IVD condition were more likely to prefer 
individual videodisc practice (x=4.58) than subjects in the VIDEO condition were to prefer 
individual videotape practice (x=3.59). The means for each question are presented in Table 5. 

Table 5 
Practice Preferences 



Preferences: VIDEO IVD 

One-on-Onc Refresher 

with BEST trainer 4.11 3.95 



Small Group refresher 

session 4.39 4.23 
Individual (Videotape/ 

Videodisc) practice 3.59 4.58 
Videotape practice at 

at home 3.63 3.95 
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There were also significant differences in the effectiveness ratings between the VIDEO and IVD 
groups. Effectiveness was measured on a five point scale ranging from 1 ineffective to 
5=Highly Effective. IVD subjects found the choice of vignettes more effective (f=3.21, df=A2, 
p=.003), the overall feedback more effective (f=3.10, df=40, p=.004) and overall, the pretest 
practice session more effective (f=3.21, #=41, p=.003). There was no significant difference in the 
effectiveness rating on the multiple-choice questions that were used. Table 6 provides the mean 
effectiveness ratings. 

Table 6 
Mean Effectiveness Ratings 

Practice components Groups 

VIDEO IVD 

Choice of Vignettes 3.58 4.28 

Multiple Choice Questions 4.05 4.17 

Overall Feedback 4.00 4.68 

Overall Effectiveness 3.95 4.54 

In addition to the effectiveness ratings reported above, subjects in the IVD condition were asked to 
rate additional aspects of practice that were unique to the IVD condition. These subjects found the 
mouse training (x=4.28) and the use of the mouse to answer questions (X=4.3o) highly effective. 
Even more effective were the instructions on the computer screens (x=4.52). Last, both the first 
form of feedback, the replay of the video segment associated with the questions (x=4.76) and the 
second form of feedback, the replay and transcript information (X=4.50) were found to be highly 
effective. 

IVD group subjects were also given an opportunity to make comments during a short debriefing 
interview. A brief summary of these comments, classified as positive or negative, are summarized 
below: 

Positive Comments 

Positive comments were generally focused on three areas; positive overall expressions about the 
training, comments about their willingness to participate if such training was available, and 
comments about the individualized nature of the feedback. 

Seventeen of '12 participants in the IVD group expressed overall satisfaction with the practice 
program. Example: "Loved it, enjoyed whole process, feel it's personal and have control over it." 

Many participants (15 out of 22) found immediate feedback of the IVD program very attractive. 
Example: "A good way of finding out what you missed", "liked the immediate feedback and 
liked the form of feedback." 
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Also, 15 out of 22 participants mentioned they would take advantage of the IVD practice program if 
such an opportunity was available. 

Several participants (5 of 22) liked the individualized nature of practice. Example: "liked personal 
control over the process", "did not feel embarrassed or intimidated as one sometimes experiences in 
group training." 

Negative Comments 

Few subjects made negative comments about the IVD practice program. The negative comments 
that were expressed focused primarily around three issues; the nature of the multiple-choice 
questions; the resemblance of the practice activity to the actual classroom and several comments 
about the hardware. 

As discussed earlier, the multiple-choice questions were designed to test the assessor's ability in 
three areas of scripting, verbatim evidence, observational comments, and critical points. Questions 
associated with observational comments were developed to encourage assessors to increase their 
observations of the classroom (e.g., what is on the blackboard, how are desks arranged, where are 
students sitting, etc.). Consequently, some assessors did not expect to encounter questions 
requiring such a close attention to classroom detail and commented accordingly. 

Seven of 22 subjects found some of the multiple-choice questions too specific or not associated with 
the most important activities found in the classroom. 

When assessors actually perform observations in classrooms they are given the opportunity to first 
interview the teacher and provided an overview of the lesson prior to the observation. Three 
comments were made that the IVD practice program did not mirror actual classroom observations: 
no preassessment information was made available; the vignettes were too short; and scripting done 
in the IVD practice differed from that found in the classroom. 

There were also several comments associated with IVD practice program itself. 

Three subjects found it sometimes difficult to hear the students. 
Two subjects had minor difficulties using the mouse. 

One subject commented that the experience was, "grueling and felt under pressure." 

Another participant suggested that other assessors might not take advantage of this system because 

they are fearful of computers. 
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Summary and Discussion 



In this pilot study we were interested in examining two major questions. First, how effective is 
the IVD approach compared to practice that involves conventional videotape procedures. 
Second, how well does the IVD systems work under field conditions. 

Answers to these questions, in part, were based on several sources of information. The 
results of the 33 item multiple-choice pretest, a 30 item post-test, an IVD and VIDEO 
pretest questionnaire and a IVD, VIDEO and CONTROL post-test questionnaire. 

As noted in the results, regardless of the condition (IVD, VIDEO or CONTROL), the 
training groups appeared to have no differential effect on performance. There are several 
likely explanations for such a finding. First, as indicated by the evidence writing scores, 
subjects involved in this study demonstrated strong scripting skills at the outset. Second, 
mean scores on both the pretest and post-test indicate that these measures were relatively 
easy. The easiness of the questions in part, was due to the use of short vignettes which may 
have capitalized on short term memory rather than scripting accuracy. Finally, given the 
information that was provided to subjects in their invitations to participate, there was no 
way of determining the an.^unt of preparation that may have taken place prior to the onset 
of the study. Each of ihese explanations, likely masked any differences that may have 
existed between the training approaches. 

Although no differences were found between training groups, there were differences 
between the pretest and post-test overall scores for both the IVD and VIDEO groups. This 
difference was largely due to the significant increase in correctly answering observational 
comment questions. This suggests that both IVD and VIDEO practice are viable training 
approaches for proficient assessors. 

As a practice program, subject evaluations favored the IVD approach. The primary reasons 
for high subject acceptance appeared due to the highly individualized, and focused 
feedback, the ability to control and replay selected video segments, and the automated 
scoring. In addition, the novelty associated with this new technology likely contributed to 
the attractiveness of this training approach. 

Evidence from this study indicates that proficient assessors are not likely to demonstrate 
large performance gains from this type of training. This is due primarily to the finding that 
assessors in this study already possessed strong scripting skills. The IVD and VIDEO 
training procedures would likely have a greater impact for those individuals first learning to 
script. 
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For individuals first learning to script, IVD training offers a number of advantages over 
VIDEO training. As we discovered, assessors preferred a method that provided 
individualized training and feedback. In addition, subject's ratings of training effectiveness 
was highest for the IVD program. It is noteworthy that participants in the IVD group 
clearly preferred IVD training over all other options. By contrast, participants in the 
VIDEO group preferred all other training options over individualized videotape practice. 

Regarding how well the IVD workstations operated in the field, we found, that in only one 
session out of 22 was there a system malfunction. In this case, a subject was simply 
rescheduled and the problem did not reappear. Based on the pilot study, we found that the 
IVD practice program required only the presence of a program administrator to start the 
program and to be available to answer questions. Consequently we are confident that IVD 
systems offer a viable approach to training where little additional human resources is 
needed. As with any hardware and software, however, problems can and will occur and 
appropriate backup systems should be in place to handle such situations. 

Based on these findings and the fact that IVD training offers administrative flexibility, cost 
effectiveness, and requires little trainer intervention, a program is now being developed to 
examine the feasibility of using IVD procedures to train beginning assessors to script. 

SECTION II: INTERACTIVE VIDEODISC SYSTEM 

This section of the paper describes and discusses each component of the IVD system utilized in the 
scripting practice study (See Figure 3). 

Videodisc 

Before detailing the IVD system, it is important to first understand why videodisc media was 
selected for this particular application. After all, it would be much easier to simply use conventional 
videotape to train assessors to practice scripting. One of the primary reasons for using IVD 
technology is based on the degree of control that can be achieved with such a format. 

The videodisc can be compared to an LP record (remember those) that can contain up to 54,000 
individual video frames (pictures) on each side of a disk. If the disc is recorded in CAV (Constant 
Angular Velocity) format, it may contain up to 30 minutes of full motion video per side. The CAV 
format operates as follows. A standard 30-minute disc maintains a constant speed of 1800 rpm 
while varying the track length for each picture. Each revolution of the disc (or track) corresponds to 
a picture containing a frame number much like the pages of a picture album. 
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it is the frame numbers that allow the capability to find and retrieve specific frames on the disc. In 
contrast to the CAV format there are extended play videodiscs that allow for as much as 60 minutes 
of full motion video per side. This longer play version of the videodisc uses a format referred to as 
constant linear velocity (CLV) which spins the disc at rates ranging from 600 to 1800 rpm. 
Unfortunately, the trade-off with the CLV format is the inability to search and play specific frame 
numbers. It is possible to search and playback only those portions of the disc that have been 
recorded with chapter numbers. Similar to the chapters of a book, these numbers allows some 
degree of search and playback capability that would not be possible with a conventional vcr. For 
purposes of this project, CAV format afforded the control necessary to provide interactive scripting 
practice. 

Most importantly, CAV format when used with a personal computer allows for random access 
capability. For example, like the hard disk drive of a personal computer, an IVD system can 
instantly search, find, and play any of the 54,000 frames. This means that unlike conventional vers 
that perform very slow and inaccurate searches, an IVD system can be programmed to provide 
nearly instantaneous search and playback that is under the control of the user. 

In addition to providing video information, the videodisc's two audio tracks can be used to reproduce 
high quality stereo or may be used be used to play in two different languages. For both the video 
and audio information, the reproduction quality is very high. 

Another attractive feature of videodiscs is that they are virtually impervious to damage by 
fingerprints, magnetism, dust, humidity and temperature. 

To produce a videodisc requires high quality video source materials. Generally speaking, to achieve 
such quality involves video recording on 3/4" to 1" videotape. In addition to the quality of the video 
tape, special editing must be done to adjust for differences that exist between videotape and 
videodisc formats. Because the pressing of the videodisc from the edited videotape requires 
specialized and costly equipment, there are presently only a few such production facilities available. 
For this project, we utilized Telstar Editing to press the videodiscs. 

Videodisc Player 

For this project, a Pioneer LD-V8000 LaserDisc player was utilized. This player was selected for its 
.5 second access time. That is, selected portions of the video on any area of the disc can be searched 
and ready for playback within .5 second. Access time is important if you want to provide immediate 
feedback to participants, as was the case in this study, or if you plan to play selected video portions 
from the same videodisk but want to give the appearance that the video is continuous. 




A variety of videodisc players are available some of which are provided below: 
Sony LDP 1000; LDP 1200; LDP 1500; LDP 2000 
Pioneer 6000, 6000A, 6010, 6020 
Pioneer 4200, 8000 

Panasonic OMDR TQ-2024; TQ-3032F 
Computer System 

IVD systems allow for a wide range of personal computers to be used. The majority of IVD 
applications currently utilize either IBM compatible (MS DOS) personal computers or the Apple line 
of computers. 

When we were first evaluating IVD systems, Apple computers required the use of two monitors. 
One monitor displayed computer text and graphics while the other monitor displayed video 
information. At that time, hardware, referred to as an overlay card, did not yet exist that would 
allow a mixing of computer and video information on a single monitor. Since overlay boards were 
available for IBM compatible computers at the time, we developed this training application around 
the IBM compatible PC. A 386sx or 386dx or higher processor is recommended as the minimum 
configuration to best take advantage of the software typically used to control an IVD system. 

Hard disk drive - Due to the size of authoring languages, text and graphics files, and related 
programs, large hard disk capacity is desirable. For this project, 40 megabyte hard drives were 
found to be satisfactory. In addition, the hard drive should be relatively fast. Avoid drives that 
report average access times longer than 23 ms. 

Graphics card - To take advantage of the features of many authoring languages as well as overlay 
cards, VGA or higher resolution graphics cards are recommended. The graphics board used in this 
application was a standard VGA Board. 

Overlay cards - For IVD systems that display computer text and graphics on the same monitor with 
video information, it is necessary to use an overlay card that can mix both forms of information. 
There are a variety of such cards available for both Apple and IBM compatibles. In this project we 
used a PSI VGAVISION I card produced by Processor Sciences, Inc. This card is not only 
compatible with the software used to control the IVD system but also allows for a variety of other 
video effects. One such effect used in this project was the compression and display of video in the 
upper quadrant of the monitor used in the second feedback screen (see Figure 1). 
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Some additional examples of VGA overlay cards include: 

IBM M-Motions Video Adapter/A 
IEV VIP-8000 

Sony SMI-3081, Sony View 5000, 3015A 
Video Logic DVA-4000 

Visage 1910 
Input Devices 

Although individuals most often use the keyboard to interact with their personal computers, 
increasingly software used to control IVD systems require a pointing device such as a mouse, a 
trackball (an upside-down mouse that is stationary), or a touch screen. 

For this project we used a Microsoft compatible mouse as an input device. Since a mouse is 
sometimes new to users, a short mouse exercise was developed to allow for practice prior to the 
start of the scripting program. 

In addition to the use of a mouse, we have been developing a speech controlled input approach as an 
alternative to conventional pointing devices. For example, in this project a Covox Voice Master Key 
speech recognition and digitizer was used. To use this system the assessor first "trains" a program to 
recognize his/her voice by repeating key commands three times. Once the system has been trained, 
an assessor is able to perform such activities as answer multiple-choice questions, control the search, 
playback and stop features of the videodisc player solely by voice commands. 

An additional feature of the Covox system that has been used in this project is the capability to 
digitally record audio information that may be played back at appropriate times. For example, when 
an assessor makes a correct or incorrect response, s/he may, in addition to a screen display, receive 
voice feedback. In the future voice feedback will be used to cue selected portions of the video. For 
example, during the observation of a video sequence, a voice signal can be used to alert an assessor 
to focus attention on an upcoming event. 

Monitors 

The choice of overlay card and graphics card will govern the type of monitor used. Since we used a 
PSI VGA VISION card, which combines the VGA and video information into a single VGA signal, 
a VGA monitor was required. 

In this project we utilized a Mitsubishi Diamond Scan monitor AUM1381A. Some of the other 
monitors compatible with an overlay IVD system include: 

InfoWindow 

Matrox 

Sony KV 1 3 1 1 , Sony K V 1 3 1 1 , Sony P VM- 1 270Q, Sony PVM - 1 27 1 Q 
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Authoring system 

An authoring language refers to a collection of programs that have been designed to assist the user 
to develop and manage a multimedia system like the IVD system discussed. Often such systems 
provide a variety of software tools that do not require computer programming skills. An example 
of the multimedia management referred to involves the process of displaying computer graphics and 
text, playing back specified portions of video based on subject responses, recording responses, and 
the like. 

Initially several authoring systems were reviewed before IconAuthor (produced by AimTech) was 
selected. At that time IconAuthor not only supported the particular hardware system that we planned 
to use but also provided a system that allowed quick and easy prototyping of the IVD application 
described in this paper. This particular authoring system provides a series of tools such as: 

Icons- a series of computer functions that are combined to build the IVD application; 

An animation module- allows users to move graphics around the computer screen; 

Color Graphics Editor- useful for drawing such objects as response buttons, etc.; 

Video Editor- allows for on screen control of the videodisc player; 

Text Editor- allows for the creation of text with a variety of fonts, sizes and colors. 

Depending on your IVD requirements, there are a wide variety of authoring lar«uages/sy stems 
available to users. For example, we understand that there are now over 1 17 such programs available. 

As our needs within the IVD system grew, we found that there were certain features of the overlay 
card that could not be addressed by the authoring system. We therefore developed programs, written 
largely in Microsoft Basic 7.1 that did provide additional capabilities. Most users, however, will 
find that many of their IVD needs can be met by full-featured authoring systems. 

Integrated Systems 

Users who prefer a prepackaged (turnkey) IVD system may wish to consider one of the following: 
IBM InfoWindow 
Matrox EIDS System 
Visage System 
Sony View System 

Summary and Conclusion 

In this section each component of an IVD system was briefly described. It is interesting to note that 
in the past, costs associated with such systems ranged from $20,000 to $30,000. Increasingly, 
computer and related hardware and software costs have been declining to the extent that IVD 
systems are becoming affordable for increasing numbers of users. 
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If you anticipate developing an IVD system, in addition to hardware and software costs, there are 
costs associated with the specialized editing of videotape and the pressing of the videodisc. The 
editing costs vary greatly depending on the number of edits and or special effects required. We 
found such costs have averaged about $1000 per 30 minutes of video. For this project we used a 
glass check disc as a low cost alternative to the typically used master videodisc. Check discs are 
cost effective in very low volumes and generally cost $400 each. A master videodisc, on the other 
hand, generally costs approximately $3000 to $4000 but quickly becomes cost effective when a large 
number of copies are produced. Each additional copy made from a master videodisc can cost as 
little as $6 each. An overall estimate of costs associated with each IVD workstation used in this 
study ranged from $9000 to $10,000. 
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Appendix A 



I. MANAGEMENT OF THE CLASSROOM ENVIRONMENT 

IA. THE TEACHER PROMOTES A POSITIVE LEARNING ENVIRONMENT 

(1) Rapport 

(2) Communication of expectations for achievement 

(3) Physical environment 

IB. THE TEACHER MAINTAINS APPROPRIATE STANDARDS OF BEHAVIOR 

(1) Rules and standards of behavior are maintained 

IC. THE TEACHER ENGAGES THE STUDENT IN THE ACTIVITIES OF THE LESSON 

( 1 ) Student engagement 

(2) Re-engagement 

ID. THE TEACHER EFFECTIVELY MANAGES ROUTINES AND TRANSITIONS 

(1) Effectiveness 

II. INSTRUCTION 

IIA. THE TEACHER CREATES A STRUCTURE FOR LEARNING 

(1) Initiations 

(2) Closures 

IIB. THE TEACHER PRESENTS APPROPRIATE LESSON CONTENT 

(1) Choice of content 

(2) Level of difficulty 

(3) Accuracy 

IIC. THE TEACHER DEVELOPS THE LESSON TO PROMOTE ACHIEVEMENT OF 
THE LESSON OBJECTIVES 

(1) Lesson Development 

(2) Use of instructional arrangements and materials 

IID. THE TEACHER USES APPROPRIATF QUESTIONING TECHNIQUES 

(1) Cognitive level 

(2) Responding to students 

(3) Opportunities for student involvement 
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IIE. THE TEACHER COMMUNICATES CLEARLY, USING PRECISE LANGUAGE AND 
ACCEPTABLE ORAL EXPRESSIONS 

(1) Precision of communication 

(2) Clarity of speech 

(3) Oral expressions 

III. ASSESSMENT 

IIIA. THE TEACHER MONITORS STUDENT UNDERSTANDING OF THE LESSON AND 
ADJUSTS INSTRUCTION WHEN NECESSARY 



( 1 ) Monitoring for understanding 

(2) Adjusting teaching when necessary 



D efining Attribute: An Example 
Indicator ID : THE TEACHER EFFECTIVELY MANAGES ROUTINES AND TRANSITIONS 

Attribute: Effectiveness 

The teacher should provide effective routines and transitions that reflect planning, established 
norms and a sense of structure. When appropriate, resources and materials should be organized and 
available. In addition, the amount of time spent on routines and transitions should be approriate for their 
purpose and makeup of the class. Depending upon the nature and purpose of a routine or transition, 
proceeding too quickly may be detrimental as taking too much time with the non-instructional activities. 
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Appendix B. 
Sample question type: Verbatim evidence 



VIGNETTE 1: 

What question does the Teacher use to initiate the discussion? 



* Do you think about the telephone as something you couldn't 
possibly live without? 




* do you think about the telephone as a nuisance that 
bothers you when you are trying to read something? 



* Do you think about the telephone as a nuisance or 
something that you couldn't possibly live without? 



Sample question type: Critical points in the lesson 



VIGNETTE 1; 



What c^oes the teacher say at the beginning of the lesson to 
establish the rules of responding to the questions? 



* Raise your hand if you have an answer. 



* Think about the question before answering, raise your hand 
if you have something to share. 



* Don't shout your answers out. 



* Think about it being a nuisance before answering, and raise 
your hand if you think you have the correct answer. 
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Sample question type: Observational comments 



VIGNETTE 1: 

Which of the following accurately describes the behavior standards 
established in the classroom? 



* Students raise hands and wait to be called on. 



* Students continually call out. 



* Students raise hands and answer. 



* Students focus their attention solely on the teacher. 
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APPENDIX C 

Beginning Educator Support and Training (BEST) Scripting Practice Questionnaire 



IVD 



Please take several minutes to complete the following questionnaire. Your comments about this laserdisc practice 
session will greatly assist us in making further improvements in the BEST assessment process. (All individual 
information about this study will remain strictly confidential.) 

Name Social Security 



1. The session in which you participated has been designed to allow you to practice your scripting skills. What other 
skills or areas within this assessment you like training or practice? 



2. Assuming each of the following opportunities was available, which of the following would you most likely auend? 
(Please check the appropriate box.) 

Unlikely to atjend Highly likely to attend 



a. 


Onc-on-one refresher with a BEST trainer 


□ 


□ 


□ 


□ 


□ 


b. 


Small group refresher session 


□ 


□ 


□ 


□ 


□ 


c. 


Individual laserdisc practice 


□ 


□ 


□ 


□ 


□ 


d. 


Videotape practice at home using tapes 
and transcripts I may check out 


□ 


□ 


□ 


□ 


□ 



3. Please rate the following components of the laserdisc practice session with regard to its effectiveness. (Please check 
the appropriate box.) 

Ineffective Highly Effective 



a. 


Mouse training 


□ 


□ 


□ 


□ 


□ 


b. 


Use of mouse to answer questions 


□ 


□ 


□ 


□ 


□ 


c. 


Instructions on the computer screens 


□ 


□ 


□ 


□ 


□ 


d. 


The multiple-choice questions used 


□ 


□ 


□ 


□ 


□ 


e. 


Choice of video vignettes 


□ 


□ 


□ 


□ 


□ 


f. 


First form of feedback-replay of the video segment 
associated with questions 


□ 


□ 


□ 


□ 


□ 


g. 


Second form of feedback-replay and transcription 
information 


□ 


□ 


□ 


□ 


□ 


h. 


Overall feedback covering the three factors 
(e.g., verbatim evidence) 


□ 


□ 


□ 


□ 


□ 


i. 


Overall, the effectiveness of this practice session 


□ 


□ 


□ 


□ 


□ 



4. Final comments about any of the above that would assist us in improving this approach to the practice of scripting: 
(Please use other side of page if necessary.) 



22 



VIDEO 



Beginning Educator Support and Training (BEST) Scripting Practice Questionnaire 

Please take several minutes to complete the following questionnaire. Your comments about this video practice session 
will greatly assist us in making further improvements in the BEST assessment process. (All individual information about 
this study will remain strictly confidential.) 

Name Social Security 



1 . The session in which you participated has been designed to allow you to practice your scripting skills. What other 
skills or areas wilhin this assessment you like training or practice? 



2. Assuming each of the following opportunities was available, which of the following would you most likely attend? 
(Please check the appropriate box.) 

Unlikely to attend H Ighly likely to attend 



a. One-on-one refresher with a BEST trainer 


□ 


□ 


□ 


n 


□ 


b. Small group refresher session 


□ 


□ 


□ 


□ 


□ 


c. Individual videotape practice 


□ 


□ 


□ 


□ 


□ 


d. Videotape practice at home using tapes 
and transcripts I may check out 


□ 


□ 


□ 


□ 


□ 


Please rate the following components of the video practice session with regard to its effectiveness. (Please che 
the appropriate box) 


Ineffective 

a. The choice of vignettes □ 


□ 


□ 


□ 


Highly Effective 
□ 


b. The multiple-choice questions used 


□ 


□ 


□ 


□ 


□ 


c. Overall feedback covering the ihrce factors 
(i.e, verbatim evidence) 


□ 


□ 


□ 


□ 


□ 


d. Overall, the effectiveness of this practice session 


□ 


□ 


□ 


□ 


□ 



4. Final comments about any of the above that would assist us in improving this approach to the practice of scripting: 
(Please use other side of page if necessary.) 
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BEST Scripting Practice Questionnaire Section 

Please take several minutes to complete the following questionnaire. Your comments about this 
session will greatly assist us in making further improvements in the BEST assessment process. 
(All individual information about this study will remain strictly confidential). 

Name Social Security No. 

1. When did you originally receive your assessor training? Month: Year: 

2. When did you receive your most recent training? Month: Year: 

3. How many assessments have you conducted to date? 

4. Briefly describe what you did to prepare yourself prior to conducting your first assigned 
assessment following the proficiency training sessions (use other side of page if needed): 



5. The session in which you participated has been designed to allow you to practice your 

scripting skills. What other skills or areas within this assessment would you like training or 
practice? 



6. Final comments about any of the above that would assist us in providing opportunities for 
practice. (Please use other side of sheet if necessary). 
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Figure 1 
Second Feedback Screen 




[Video Controller] 

[ ] Play Videodisc 

[ ] Stop Videodisc 

[ ] Rewind 

[ ] Return to Question 



[Script Controller] 

[ ] Page Up 
[ ] Page Down 

Page 1 of 1 



[Script] 

T: Okay, listen. I want you to think about this question, think 
about it before you answer, and then raise your hand if you 
have something to share. Do you think... 



Figure 2 

Interactive Videodisc Practice and Questioning Process 



VIEWING OF VIDEO "VIGNETTES" 

Assessor views two video vignettes and scripts 
classroom observations and talk 



"INTERACTIVE" QUESTIONING & TRAINING 

Immediately following the viewing of the vignette the assessor 
is asked to answer multiple-choice questions. 





Is this 
the first incorrect' 
response to the 
same question ? 

NO V X YES 



Go to 
the next 
question 





Produce 




score 




report 



1. 

Replay Video 



View transcript 



Replay relevant 
video to same 
question 



END 


t 


Second feedback screen 

1. Replay Video 

2. View Transcript 

3. Return to question 





3. - Return to 
same question 
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Figure 3 

Components of the Interactive Videodisc System 
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